Researchers from Meta have been teasing a new optimizer on X. They have now released the code along with various integrations. The optimizer has no LR schedule, which means you don’t need to know the full number of training steps beforehand. It has been shown empirically to work on a wide variety of problems including language models.
Monday, April 8, 2024ReFT is a new parameter-efficient method for fine-tuning language models. It is substantially cheaper than even PeFT while achieving strong performance.
Node.js 22 contains highlights such as require()ing ES modules, a WebSocket client, updates of the V8 JavaScript engine, and more. It will enter long-term support in October. This blog post expands on the changelog for the release, adding additional context to the changes. Node.js 18 will go End-of-Life in April 2025.